A real-world evaluation of the implementation of NLP technology in abstract screening of a systematic review.

The laborious and time-consuming nature of systematic review production hinders the dissemination of up-to-date evidence synthesis. Well-performing natural language processing (NLP) tools for systematic reviews have been developed, showing promise to improve efficiency. However, the feasibility and value of these technologies have not been comprehensively demonstrated in a real-world review. We developed an NLP-assisted abstract screening tool that provides text inclusion recommendations, keyword highlights, and visual context cues. We evaluated this tool in a living systematic review on SARS-CoV-2 seroprevalence, conducting a quality improvement assessment of screening with and without the tool. We evaluated changes to abstract screening speed, screening accuracy, characteristics of included texts, and user satisfaction. The tool improved efficiency, reducing screening time per abstract by 45.9% and decreasing inter-reviewer conflict rates. The tool conserved precision of article inclusion (positive predictive value; 0.92 with tool vs. 0.88 without) and recall (sensitivity; 0.90 vs. 0.81). The summary statistics of included studies were similar with and without the tool. Users were satisfied with the tool (mean satisfaction score of 4.2/5). We evaluated an abstract screening process where one human reviewer was replaced with the tool's votes, finding that this maintained recall (0.92 one-person, one-tool vs. 0.90 two tool-assisted humans) and precision (0.91 vs. 0.92) while reducing screening time by 70%. Implementing an NLP tool in this living systematic review improved efficiency, maintained accuracy, and was well-received by researchers, demonstrating the real-world effectiveness of NLP in expediting evidence synthesis.

Global seroprevalence of SARS-CoV-2 antibodies: A systematic review and meta-analysis.

BACKGROUND: Many studies report the seroprevalence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibodies. We aimed to synthesize seroprevalence data to better estimate the level and distribution of SARS-CoV-2 infection, identify high-risk groups, and inform public health decision making. METHODS: In this systematic review and meta-analysis, we searched publication databases, preprint servers, and grey literature sources for seroepidemiological study reports, from January 1, 2020 to December 31, 2020. We included studies that reported a sample size, study date, location, and seroprevalence estimate. We corrected estimates for imperfect test accuracy with Bayesian measurement error models, conducted meta-analysis to identify demographic differences in the prevalence of SARS-CoV-2 antibodies, and meta-regression to identify study-level factors associated with seroprevalence. We compared region-specific seroprevalence data to confirmed cumulative incidence. PROSPERO: CRD42020183634. RESULTS: We identified 968 seroprevalence studies including 9.3 million participants in 74 countries. There were 472 studies (49%) at low or moderate risk of bias. Seroprevalence was low in the general population (median 4.5%, IQR 2.4-8.4%); however, it varied widely in specific populations from low (0.6% perinatal) to high (59% persons in assisted living and long-term care facilities). Median seroprevalence also varied by Global Burden of Disease region, from 0.6% in Southeast Asia, East Asia and Oceania to 19.5% in Sub-Saharan Africa (p<0.001). National studies had lower seroprevalence estimates than regional and local studies (p<0.001). Compared to Caucasian persons, Black persons (prevalence ratio [RR] 3.37, 95% CI 2.64-4.29), Asian persons (RR 2.47, 95% CI 1.96-3.11), Indigenous persons (RR 5.47, 95% CI 1.01-32.6), and multi-racial persons (RR 1.89, 95% CI 1.60-2.24) were more likely to be seropositive. Seroprevalence was higher among people ages 18-64 compared to 65 and over (RR 1.27, 95% CI 1.11-1.45). Health care workers in contact with infected persons had a 2.10 times (95% CI 1.28-3.44) higher risk compared to health care workers without known contact. There was no difference in seroprevalence between sex groups. Seroprevalence estimates from national studies were a median 18.1 times (IQR 5.9-38.7) higher than the corresponding SARS-CoV-2 cumulative incidence, but there was large variation between Global Burden of Disease regions from 6.7 in South Asia to 602.5 in Sub-Saharan Africa. Notable methodological limitations of serosurveys included absent reporting of test information, no statistical correction for demographics or test sensitivity and specificity, use of non-probability sampling and use of non-representative sample frames. DISCUSSION: Most of the population remains susceptible to SARS-CoV-2 infection. Public health measures must be improved to protect disproportionately affected groups, including racial and ethnic minorities, until vaccine-derived herd immunity is achieved. Improvements in serosurvey design and reporting are needed for ongoing monitoring of infection prevalence and the pandemic response.

The Sensitivity and Costs of Testing for SARS-CoV-2 Infection With Saliva Versus Nasopharyngeal Swabs : A Systematic Review and Meta-analysis.

BACKGROUND: Nasopharyngeal swabs are the primary sampling method used for detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), but they require a trained health care professional and extensive personal protective equipment. PURPOSE: To determine the difference in sensitivity for SARS-CoV-2 detection between nasopharyngeal swabs and saliva and estimate the incremental cost per additional SARS-CoV-2 infection detected with nasopharyngeal swabs. DATA SOURCES: Embase, Medline, medRxiv, and bioRxiv were searched from 1 January to 1 November 2020. Cost inputs were from nationally representative sources in Canada and were converted to 2020 U.S. dollars. STUDY SELECTION: Studies including at least 5 paired nasopharyngeal swab and saliva samples and reporting diagnostic accuracy for SARS-CoV-2 detection. DATA EXTRACTION: Data were independently extracted using standardized forms, and study quality was assessed using QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2). DATA SYNTHESIS: Thirty-seven studies with 7332 paired samples were included. Against a reference standard of a positive result on either sample, the sensitivity of saliva was 3.4 percentage points lower (95% CI, 9.9 percentage points lower to 3.1 percentage points higher) than that of nasopharyngeal swabs. Among persons with previously confirmed SARS-CoV-2 infection, saliva's sensitivity was 1.5 percentage points higher (CI, 7.3 percentage points lower to 10.3 percentage points higher) than that of nasopharyngeal swabs. Among persons without a previous SARS-CoV-2 diagnosis, saliva was 7.9 percentage points less (CI, 14.7 percentage points less to 0.8 percentage point more) sensitive. In this subgroup, if testing 100 000 persons with a SARS-CoV-2 prevalence of 1%, nasopharyngeal swabs would detect 79 more (95% uncertainty interval, 5 fewer to 166 more) persons with SARS-CoV-2 than saliva, but with an incremental cost per additional infection detected of $8093. LIMITATION: The reference standard was imperfect, and saliva collection procedures varied. CONCLUSION: Saliva sampling seems to be a similarly sensitive and less costly alternative that could replace nasopharyngeal swabs for collection of clinical samples for SARS-CoV-2 testing. PRIMARY FUNDING SOURCE: McGill Interdisciplinary Initiative in Infection and Immunity. (PROSPERO: CRD42020203415).

Proportion of asymptomatic infection among COVID-19 positive persons and their transmission potential: A systematic review and meta-analysis.

BACKGROUND: The study objective was to conduct a systematic review and meta-analysis on the proportion of asymptomatic infection among coronavirus disease 2019 (COVID-19) positive persons and their transmission potential. METHODS: We searched Embase, Medline, bioRxiv, and medRxiv up to 22 June 2020. We included cohorts or cross-sectional studies which systematically tested populations regardless of symptoms for COVID-19, or case series of any size reporting contact investigations of asymptomatic index patients. Two reviewers independently extracted data and assessed quality using pre-specified criteria. Only moderate/high quality studies were included. The main outcomes were proportion of asymptomatic infection among COVID-19 positive persons at testing and through follow-up, and secondary attack rate among close contacts of asymptomatic index patients. A qualitative synthesis was performed. Where appropriate, data were pooled using random effects meta-analysis to estimate proportions and 95% confidence intervals (95% CI). RESULTS: Of 6,137 identified studies, 71 underwent quality assessment after full text review, and 28 were high/moderate quality and were included. In two general population studies, the proportion of asymptomatic COVID-19 infection at time of testing was 20% and 75%, respectively; among three studies in contacts it was 8.2% to 50%. In meta-analysis, the proportion (95% CI) of asymptomatic COVID-19 infection in obstetric patients was 95% (45% to 100%) of which 59% (49% to 68%) remained asymptomatic through follow-up; among nursing home residents, the proportion was 54% (42% to 65%) of which 28% (13% to 50%) remained asymptomatic through follow-up. Transmission studies were too heterogenous to meta-analyse. Among five transmission studies, 18 of 96 (18.8%) close contacts exposed to asymptomatic index patients were COVID-19 positive. CONCLUSIONS: Despite study heterogeneity, the proportion of asymptomatic infection among COVID-19 positive persons appears high and transmission potential seems substantial. To further our understanding, high quality studies in representative general population samples are required.